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ABSTRACT 


Five levels of autopilot complexity were flown in a single engine IFR 
simulation for several different IFR terminal operations. A comparison was 
made of the five levels of complexity ranging from no autopilot to a fully 
coupled lateral and vertical guidance mode to determine the relative benefits 
versus complexity/cost of state-of-the-art autopilot capability in the IFR 
terminal area. Of the five levels tested, the heading select mode made the 
largest relative difference in decreasing workload and simplifying the 
approach task. It was also found that the largest number of blunders was 
detected with the most highly automated mode. The data also showed that, 
regardless of the autopilot mode, performance during an IFR approach was highly 
dependent on the type of approach being flown. These results indicate that 
automation can be useful when making IFR approaches in a high workload environ- 
ment, but also that some disturbing trends are associated with some of the 
higher levels of automation found in state-of-the-art autopilots. 
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Seven subjects were used in the tests, two NASA test pilots and five IFR 
rated pilots with various levels of IFR and autopilot experience. Each subject 
flew 27 data runs, for a total of 189 runs for this study. This included the 25 
different combinations of five autopilot modes and five different approaches. 

The extra two runs per subject were repeats for replication purposes. The order 
of presentation was randomly determined for each pilot. Each data run lasted 
from 10 to 20 minutes, depending on the specific approach being flown. The 
ceiling and visibility for each run were randomly chosen from three conditions 
predefined for each of the five approaches. They were: (1) 15.2 m (50 ft) 
ceiling and 0.8 km (0.5 mi) visibility for the given approach, (2) published 
minimums for the given approach, or (3) 61 m (200 ft) above ceiling and double 
visibility of published minimums for the given approach. All the runs were 
flown in moderate turbulence (1.2 m/sec (4 ft/sec)) and 20 kt winds from a 
predefined direction. The piloting task consisted of flying the specified 
approach, making the required pilot reports, and performing a side task. 


EXPERIMENTAL DESIGN 

o SUBJECTS (7) 
o NUMBER OF RUNS (189) 
o RUN LENGTH (10-20 MIN) 
o WEATHER (BELOW MIN, AT MIN, ABOVE MIN) 
o TASK 
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Five levels of autopilot automation were tested. The five, in order of 
increasing levels of automation, consisted of 1) no autopilot (NA - the basic 
aircraft); 2) wing leveler (WL) ; 3) heading select (HS - a heading select 
directional gyro was used in this mode) ; 4) heading select with lateral 
navigation coupling (HC - this mode included lateral guidance for both very 
high frequency omni range (VOR) and instrument landing system (ILS) navigation) ; 
and 5) heading select with lateral navigation coupling and altitude hold with 
vertical navigation coupling (HAC) . In addition to the previously discussed 
capabilities this mode also included a choice of pitch attitude hold, altitude 
hold, or vertical navigation guidance (i.e., glideslope coupler). 


AUTOPILOT MODES 


o 

NA 

no autopilot 




o 

WL 

wing leveler 




o 

HS 

heading select 




o 

HC 

heading select 

with 

lateral 

nav coupler 

o 

HAC 

heading select 

with 

lateral 

nav coupler and altitude hold with vertical 


nav coupler 
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Five airports and their associated radio nav aids located in the general 
vicinity of Langley Research Center were programmed and used in this study 
The types of approaches included two ILS approaches, one VOR approach, one Loc 
BC approach, and one NDB approach* These approaches, and other pertinent 
information, are given in more detail below. 


APPROACHES 


Airport 

Runway 

Approaches 

Display 

Wind 

Norfolk, VA 

5 

ILS 

CDI 

091°/20 kt 

Atlanta, GA 

8 

ILS 

CDI 

225°/20 kt 

Newport News, VA 

25 

Loc/BC (Holding) CDI 

290°/20 kt 

Franklin, VA 

9 

VOR 

CDI 

332°/20 kt 

Wakefield, VA 

20 

NDB 

Fixed compass card 

155°/20 kt 


(From ref. 1 . ) 



The data taken during each approach consisted of flight technical error, 
ground track and profile plots, pilot workload rating and comments, and side 
task results. 


DATA 

o FLIGHT TECHNICAL ERROR 
o GROUND TRACK AND PROFILE PILOTS 
o PILOT WORKLOAD RATINGS 
o PILOT COMMENTS 


o SIDE TASK RESULTS 



The side task results, in general, are representative of all the data. 

This figure shows the average number of problems completed per run during all 
the approaches for all the subjects at each level of autopilot complexity. The 
upper and lower limit bars represent the maximum and minimum of the averages of 
the individual subjects at each level of autopilot complexity. Implicit in using 
a secondary task is the assumption that the more difficult the task, the fewer 
problems completed, hence the higher the workload associated with the primary 
task. As can be seen by the data, the workload tends to decrease (increased 
secondary task performance) as automation level is increased. Significant, 
however, is the leveling off of the workload for automation levels greater than 
the HS mode. One interpretation of this phenomenon is that beyond the HS mode 
the subject trades off the workload associated with flying the control task for 
the workload required to monitor the autopilot's control of the flight task. 

This results in little net difference in primary task workload beyond the HS mode. 


Average number of side tasks 
(From ref. 1.) 
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This figure shows a similar relationship with respect to subjective pilot 
workload ratings. At the end of each run the subject rated the primary task on 
a workload scale of 1 to 7 with 1 designated as the easiest and 7 as the hardest. 
It should be realized that this type of rating technique typically produces a 
relative workload rating of difficulty rather than an absolute workload rating. 
The format of this figure is similar to that of the previous figure; i.e., shown 
is the average workload rating per run during all the approaches for all the 
subjects at each level of autopilot complexity. The upper and lower limit bars 
represent the maximum and minimum of the averages of the individual subjects at 
each level of autopilot complexity. These results tend to agree with the side 
task results; i.e., increased automation decreases workload. There is also a 
slight leveling off of the workload beyond the HS mode, but it is not as dramatic 
as in the side task data. 


Workload ratings 
(From ref. 1.) 
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Several disturbing trends were noted as the level of autopilot automation 
was increased. In general, an increased level of automation tends to take 
the pilot out of the aircraft control loop. He becomes a manager of the auto- 
pilot functions. The effects of this change in duty appear to be emphasized 
in the HAC mode. The subjects were more likely to lose track of where they 
were in the approach. It seemed that in monitoring the autopilot the pilot would 
associate instrument readings with the autopilot functions rather than with 
situational awareness. Therefore, if the autopilot functions were either 
set incorrectly or interpreted incorrectly, the subject would frequently 
perform the wrong task, thinking that everything was normal. This would 
frequently lead to an incident or blunder. An example is shown below 
(Franklin VOR approach, HAC mode). The run began with the autopilot set in 
the heading select mode. After crossing the VOR, a right turn to the 
outbound course was initiated. At this point the autopilot was switched to 
omni coupler to intercept and track the outbound course. However, the subject 
had neglected to reset the correct bearing on the CDI. Therefore, the 
autopilot reintercepted and tracked the original bearing of the CDI. 

Eventually, he realized his mistake and set the correct outbound bearing on the 
CDI. The aircraft then took up a 45° intercept path to the new bearing. After 
a fair amount of time he still had not intercepted the outbound course turn using 
to the time into the approach he decided to make a pseudo procedure turn using 
heading select. At this point in time he also set in the correct inbound 
heading on the CDI. Upon completion of the procedure turn he continued in 
heading select until the CDI needle came alive. He then selected omni coupler 
and completed the approach without further incident. It is likely this incident 
would not have been detected in the real world. 


DISTANCE 
(n. ml. ) 


RUNWAY 



WIND 
20 kts 


ALTITUDE HOLD 
ALTITUDE HOLD 
HEADING SELECT 
PITCH COMMAND 
FRANKLIN VOR 
PITCH COMMAND 
OMNI 

ALTITUDE HOLD 

ALTITUDE HOLD 
PITCH COMMAND 
OMNI 

HEADING SELECT 


-2 0 

DISTANCE (n. mi.) 


Ground track Franklin VOR approach. 

(From ref. 1.) 


HAC autopilot mode. 



Another subject (Wakefield NDB approach, HAC mode) made his 
final let down on an outbound heading. He leveled off and made his missed 
approach without ever realizing his mistake. Another interesting facet 
related to this run is the fact that the NDB at Wakefield is located on the 
airport. The missed approach should have been executed when, if in this case 
the NDB was crossed. In fact several otherwise normal runs were also flown 
at Wakefield in which the missed approach was executed prior to crossing the 
NDB inbound. It seems that the subjects would time their outbound leg and 
use this time, rather than the NDB crossing, to execute their missed approach 
The 45° left headwind on the inbound heading was obviously a contributing 
factor in these incidents. These results imply a lack of positional 
awareness. 


GROUNDTRACK ALTITUDE PROFILE 



Wakefield NDB approach. HAC autopilot mode. 
(From ref. 1.) 


The results of this study suggests several general implications. 

Automation can reduce pilot workload, but a poor pilot interface with 
complex levels of automation can lead to disastrous blunders. In general, 
an increased level of automation tends to take the pilot out of the aircraft 
control loop. He becomes a manager of the autopilot functions. It seemed that 
in monitoring the autopilot the pilot would associate instrument readings with 
the autopilot functions rather than with situational awareness. The problem 
appears to be almost as if the pilot thinks of the autopilot as a copilot and 
expects it to think for itself. He allows himself to become completely 
engrossed in other tasks once the autopilot is set. Hence, he is frequently 
late in resetting new functions or he may become confused as to exactly where 
he is in the approach and not reset all the necessary functions or controls. 


IMPLICATIONS 

o AUTOMATION IS BENEFICIAL "BUT" 
o PILOT BECOMES AUTOPILOT MANAGER 
o LOSS OF SITUATIONAL AWARENESS 
o AUTOPILOT/COPILOT 
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The results of this study indicate that automation is desirable when 
making IFR approaches in a high workload environment, but also that some 
disturbing trends are associated with the higher levels of automation as 
presently implemented in state-of-the-art autopilots. It is believed, 
however, that a better man/machine interface could alleviate these problems. 

The data further suggest that the heading select mode may currently be the 
best choice for the IFR approach task when considering both benefits and costs. 


CONCLUSIONS 


o AUTOMATION DECREASES WORKLOAD. 

o THE MEASURED WORKLOAD BEGAN LEVELING OFF AT THE HEADING SELECT MODE. 

o THE LARGEST INCREMENT OF BENEFIT WAS OBTAINED WITH THE HEADING SELECT MODE. 

o THE MAJORITY OF THE BLUNDERS OCCURRED WITH THE MORE HIGHLY AUTOMATED MODES. 

o AUTOMATION IS BENEFICIAL BUT CAN LEAD TO PROBLEMS IF NOT JUDICIOUSLY 
INTERFACED WITH THE PILOT. 
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